Skip to content

Merge changes from upstream - #810

Merged
tgilon merged 13 commits into
masterfrom
merge-upstream-20260727
Jul 30, 2026
Merged

Merge changes from upstream#810
tgilon merged 13 commits into
masterfrom
merge-upstream-20260727

Conversation

@tgilon

@tgilon tgilon commented Jul 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Closes # (if applicable).

Changes proposed in this Pull Request

This PR brings the latest changes from PyPSA/PyPSA-Eur. Among other things, this PR introduces the following:

Merge conflicts:

  • doc/release_notes.md
  • envs/default_linux-64.pin.txt
  • envs/default_osx-64.pin.txt
  • envs/default_osx-arm64.pin.txt
  • envs/default_win-64.pin.txt
  • pixi.lock
  • pixi.toml

Tasks

Workflow

Open issues

Notes

Checklist

Required:

  • Security scans show no high-severity bugs, critical vulnerabilities, or exposed secrets.
  • Changes are tested locally and behave as expected.
  • Code and workflow changes are documented.
  • A release note entry is added to doc/release_notes.md.
  • The description is human-written and any AI-generated content is marked.

If applicable:

  • Changes in configuration options are reflected in scripts/lib/validation.
  • Changes in configuration options are added to config/test/*.yaml.
  • Multiple climate years test passes locally (pixi run -e open-tyndp tyndp-cyears-test).
  • For new data sources or versions, these instructions have been followed.
  • Open-TYNDP SPDX license header is added to all touched files.
  • Module docstrings are added to new Python scripts.
  • New rules are documented in the appropriate doc/*.md files.
  • Major features are documented in doc/index.md.

daniel-rdt and others added 11 commits July 9, 2026 15:00
@daniel-rdt
fix: add missing regular expression anchor with re.fullmatch (PyPSA#2225
d1672f9 
)

* fix: add missing regular expression anchors with re.fullmatch

* doc: add release note
@tgilon
Merge branch 'PyPSA:master' into master
3139c78
@FabianHofmann @pypsa
chore: pin linopy 0.8.0 and override exclude-newer for pypsa/linopy/a…
308c739 
…tlite (PyPSA#2237)

* chore: pin linopy 0.8.0 and override exclude-newer

Bump linopy to ==0.8.0 and add a per-package exclude-newer override
so the freshly released version is visible to the solver. Relock.

* [pypsa-bot] run `sync-locks`

- `pixi run sync-locks`

* add pypsa and atlite to non-cooldown setup

* relax linopy version to 0.8.x

* [pypsa-bot] run `sync-locks`

- `pixi run sync-locks`

---------

Co-authored-by: pypsa[bot] <181215446+pypsa[bot]@users.noreply.github.com>
@hd1ex
Fix Ultranet (TYNDP 254) to end in Philippsburg (PyPSA#2236)
18fc1ca 
Until now, this HVDC link falsely ended in Philippsbourg (~70 km west).
With PyPSA-Eur's clustering enabled (which respects country borders),
this leads to an HVDC connection with France leading to wrong results.
@mvanhorn
fix: add retries to WDPA/WDPA-marine download rules (PyPSA#2238)
e9a1fb5 
Fixes PyPSA#2138

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
@w-tusche
fix: change linting and formatting documentation to correctly show ru…
7c35b05 
…ff (PyPSA#2233)
@daniel-rdt @fneum
fix: define inflow_t before branch so it is always assigned (PyPSA#2224)
85c9736 
* fix: define inflow_t before branch so it is always assigned

* doc: add release note

---------

Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
@euronion @fneum
doc: Add FAQ section for certificate errors (PyPSA#2228)
0cbacfc 
* doc: Add FAQ section for certificate errors

* Add release notes

---------

Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>
@tgilon
Merge branch 'PyPSA:master' into master
df6d4f7
@tgilon
Merge branch 'master-upstream-oet'
687799b 
 Conflicts:
	doc/release_notes.md
	envs/default_linux-64.pin.txt
	envs/default_osx-64.pin.txt
	envs/default_osx-arm64.pin.txt
	envs/default_win-64.pin.txt
	pixi.lock
	pixi.toml
@tgilon
chore: update pixi env and sync locks
d4b3acb
@tgilon tgilon added this to the Release v0.8 milestone Jul 27, 2026
@tgilon tgilon self-assigned this Jul 27, 2026
@tgilon tgilon added SB Scenario Building CBA Cost Benefit Analysis labels Jul 27, 2026
@tgilon
tgilon marked this pull request as ready for review July 27, 2026 14:31
@daniel-rdt

daniel-rdt commented Jul 29, 2026

Copy link
Copy Markdown
Member

@tgilon
cc: @measrainsey @lisazeyen
I ran a high-resolution test on this PR branch and compared it with the previous run NT-1H-20260703-pr-sort-links from our previous upstream-merge (see details for the config).

Details 
NT-cy2009-20260727-upstream-merge:
  tyndp_scenario: NT
  clustering:
    temporal:
      resolution_sector: 1H

  solving:
    options:
      assign_all_duals: true
    solver:
      name: gurobi
      options: gurobi-default
      solver_options:
        gurobi-default:
          BarConvTol: 1.0e-06
          FeasibilityTol: 1.0e-06
          OptimalityTol: 1.0e-06
    partition: big
    mem_mb: 60000
    runtime: 1h

  cba:
    projects: [t4]

    msv_extraction:
      solving:
        options:
          include_objective_constant: false
        solver:
          name: gurobi
          options: gurobi-default
        solver_options:
          gurobi-default:
            BarConvTol: 1.0e-06
            FeasibilityTol: 1.0e-05
            OptimalityTol: 1.0e-06
        partition: big
        mem_mb: 60000
        runtime: 1h

    # Rolling horizon solving settings
    solving:
      options:
        include_objective_constant: false
      partition: big
      mem_mb: 60000
      runtime: 1h

      solver:
        name: highs
        options: highs-simplex

      solver_options:
        highs-simplex:
          primal_feasibility_tolerance: 1.0e-06
          dual_feasibility_tolerance: 1.0e-06

Here are some system-level comparisons first:

Expanded capacity

image

Link comparisons

Only a few links differ in marginal cost:
image

Total Opex and objective

image image

Load Shedding

image

My main feeling is that the results all look plausible and very very similar to before. But tightening numerical tolerances for the new run creates some visible shifts in the results when we look into the details.

Analogue to the previous investigation, I created some figures to visualise the deviations >1% and >1 GWh for the solved SB networks. In the details, you'll also find a scatter plot with all the data points.

Energy balance - AC

image
Details image

Energy balance - H2

image
Details image

Transmission - AC

image
Details image

Transmission - H2

image
Details image

Opex - AC

image
Details image

Opex - H2

image
Details image

@tgilon

tgilon commented Jul 30, 2026

Copy link
Copy Markdown
Member Author

@tgilon cc: @measrainsey @lisazeyen I ran a high-resolution test on this PR branch and compared it with the previous run NT-1H-20260703-pr-sort-links from our previous upstream-merge (see details for the config).

Thank you for the detailed comparisons. I agree with you that the results are reasonable and that the deviations are minor. This is sufficient for me and I'm happy to keep the current versions (PyPSA / Linopy).

@tgilon
tgilon requested a review from daniel-rdt July 30, 2026 08:18
@daniel-rdt

daniel-rdt commented Jul 30, 2026

Copy link
Copy Markdown
Member

@tgilon cc: @measrainsey @lisazeyen I ran a high-resolution test on this PR branch and compared it with the previous run NT-1H-20260703-pr-sort-links from our previous upstream-merge (see details for the config).

Thank you for the detailed comparisons. I agree with you that the results are reasonable and that the deviations are minor. This is sufficient for me and I'm happy to keep the current versions (PyPSA / Linopy).

Great, then we can move on :) I think we should open a follow up PR though to specify the tolerances for SB and CBA we figured out along the way, don't you think?

@tgilon tgilon mentioned this pull request Jul 30, 2026
Closed
@tgilon

tgilon commented Jul 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Great, then we can move on :) I think we should open a follow up PR though to specify the tolerances for SB and CBA we figured out along the way, don't you think?

That makes a lot of sense. I'll let you do it in your solver updates PR. Anything left to add to this PR before merging?

daniel-rdt
daniel-rdt approved these changes Jul 30, 2026
View reviewed changes

@daniel-rdt daniel-rdt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anything left to add to this PR before merging?

Don't think so. Not many other changes in this merge. Looking good to me.

@tgilon
tgilon merged commit cedff48 into master Jul 30, 2026
8 of 9 checks passed
@tgilon
tgilon deleted the merge-upstream-20260727 branch July 30, 2026 14:10
@tgilon tgilon mentioned this pull request Jul 31, 2026
Closed
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@daniel-rdt daniel-rdt daniel-rdt approved these changes

Assignees

@tgilon tgilon

Labels

CBA Cost Benefit Analysis SB Scenario Building

Projects

None yet

Milestone

Release v0.8

Development

Successfully merging this pull request may close these issues.

7 participants

@tgilon @daniel-rdt @FabianHofmann @hd1ex @mvanhorn @w-tusche @euronion